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he  Institute  of  Electrical  and  Electron¬ 
ics  Engineers  (IEEE)  802.16  standard 
(IEEE  2004;  IEEE  2006),  also  known 
as  Worldwide  Interoperability  for  Mi¬ 
crowave  Access  (WiMAX),  includes 
specifications  for  fixed  and  mobile  broadband  wireless 
access  (BWA).  Deployment  of  WiMAX  is  particularly 
attractive  in  tactical  military  settings  where  “cells”  of 
high-data  rate  wireless  connectivity  must  be  established 
rapidly  and  relatively  inexpensively  (see  Figure  T). 

The  802.16  standard  prescribes  several  quality-of- 
service  (QoS)  classes,  which  help  ensure  the  reliability 
and  timeliness  of  critical  tactical  edge  (TE)  applica¬ 
tions  such  as  voice  over  internet  protocol  (VoIP),  real¬ 
time  situational  awareness  (SA),  and  command  and 
control  (C2). 

A  typical  configuration  for  WiMAX,  called  point- 
to-multipoint  (PMP)  mode,  involves  two  types  of 
communication  stations:  (a)  base  station  (BS)  and  (b) 
subscriber  stations  (SS).  The  BS  (perhaps  located  at 
the  command  center)  regulates  all  communication  in 
the  ceh  network.  Data  are  transmitted  from  the  SS  to 
the  BS  in  the  uplink  direction  and  from  the  BS  to  the 
SS  in  the  downlink  direction.  Sagacious  allocation  of 
available  bandwidth  “slots”  in  the  uplink  direction  is 
crucial  to  QoS  of  WiMAX  at  the  TE. 

The  amount  of  bandwidth  each  SS  is  ahowed  to 
have  in  the  uplink  direction  is  dynamically  determined 
by  the  BS  in  the  form  of  an  “uplink  scheduling 
algorithm”  (Khalil  and  Ksentini  2007;  Belghith  and 
Nuaymi  2008).  This  algorithm  is  not  specified  in  the 


802.16  standard,  thus  giving  WiMAX  implementers 
the  option  of  choosing,  or  even  designing,  optimized 
uplink  schedulers  that  meet  specific  needs.  For 
instance,  one  may  seek  to  maximize  the  system 
throughput  while  maximizing  the  number  of  trans¬ 
mitted  data  packets  with  hard  deadlines.  An  efficient 
uplink  scheduling  algorithm  at  the  TE  should  consider 
the  QoS  constraints  imposed  by  characteristic  TE 
application  traffic  while  seeking  to  maximize  the 
throughput  of  the  system  (Yu  2008;  Pishdad  and  Rabiee 
2008;  Piro  et  al.  2010;  Wongthavarawat  and  Ganz  2003; 
Mohammadi,  Akl,  and  Behnamfar  2008b). 


Figure  1.  Simple  tactical-edge  Worldwide  Interoperability  for 
Microwave  Access  (WiMAX)  depiction. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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Approaches 

Test  and  evaluation  (T&E)  of  WiMAX  QoS  using 
existing  and  proposed  uplink  scheduling  algorithms 
requires  an  environment  flexible  enough  to  select  and 
implement  different  uplink  schedulers.  A  field  test 
with  physical  hardware  utilizing  field-programmable 
gate  arrays  (FPGAs)  with  the  ability  to  reprogram  new 
uplink  schedulers  is  theoretically  possible.  Currently, 
however,  most  equipment  used  for  WiMAX  field  tests 
within  the  Department  of  Defense  (DoD)  do  not 
contain  FPGA  components.  In  addition,  few  testers 
possess  the  requisite  expertise  with  hardware  descrip¬ 
tion  languages  (HDLs)  and  programing  tools  in  order 
to  program  FPGAs.  Even  given  FPGA-endowed 
equipment,  HDL  tools,  and  programmer  resources, 
field  testing  of  alternative  WiMAX  QoS  configura¬ 
tions  and  algorithms  would  be  cumbersome  and  costly. 

A  more  flexible,  rapid,  and  cost-effective  way  to  test 
and  evaluate  WiMAX  QoS  is  to  use  discrete  event 
simulation,  a  high-fidelity  form  of  modeling  and 
simulation  (M&S).  A  discrete-event  network  simulator 
emulates  the  behavior  of  an  interconnected  network 
and  applications,  including  detailed  processing  through 
all  layers  of  the  protocol  stack.  In  the  case  of  WiMAX, 
this  should  include  an  accurate  media  access  control 
(MAC)  and  physical  layer  (PHY)  implementation  of 
the  802. 16d  and  802. 16e  specifications  with  PMP 
mode  and  the  Wireless  Metropolitan  Area  Network 
Orthogonal  Frequency-Division  Multiplexing  (MAN- 
OFDM)  PHY  layer.  These  two  specifications  are  also 
known  as  fixed  and  mobile  wireless,  respectively. 
Examples  of  network  simulators  that  claim  to  accu¬ 
rately  implement  the  802.16d/e  standards  include  ns- 
2/ns-3  (NSNAM),  OPNET  (OPNET  Technologies, 
Inc.),  and  QualNet  (Scalable  Network  Technologies). 

Two  important  requirements  for  our  T&E  of 
WiMAX  QoS  with  new  and  existing  uplink  schedul¬ 
ing  algorithms  are  cost  and  modifiability.  The  two 
simulators  ns-2  and  ns-3  are  open-source.  For 
comparison,  OPNET  is  proprietary  software  requiring 
the  purchase  of  multiple  licenses  and  support  options  at 
a  current  cost  approaching  $60K.  An  integrated 
graphical  user  interface  (GUI)  is  lacking  in  ns-2/ns- 
3,  requiring  C/C++  programming  in  order  to  configure 
the  simulation.  OPNET,  on  the  other  hand,  possesses 
a  mature  and  intuitive  GUI  for  configuration. 

Cost  and  ease  of  configuration  aside,  the  greatest 
argument  for  using  an  open-source  simulator  is  the 
ability  to  modify  and  add  source  code.  T&E  of  existing 
and  proposed  uplink  scheduling  algorithms  requires 
the  ability  to  incorporate  a  significant  amount  of  logic 
into  the  simulator  that  goes  beyond  simple  configura¬ 
tion.  Proprietary  M&S  tools  may  offer  the  user  a 
limited  ability  to  configure  “built-in”  algorithms  or  add 


some  process  code  but  often  provide  no  mechanism  to 
significantly  modify  existing  algorithms  or  implement 
new  algorithms.  When  such  a  provision  is  available, 
the  modification  or  implementation  of  the  algorithm  is 
realized  as  C  or  C++  code. 

Solution  components 

We  propose  and  utilize  integrated  solution  archi¬ 
tecture  for  T&E  of  WiMAX  performance  on  QoS- 
constrained  TE  traffic  (see  Figure  2).  The  solution 
represents  a  loose  coupling  of  five  open-source  and 
freeware  software  tools.  We  briefly  describe  each  of  the 
five  tools  in  this  section  and,  where  relevant,  address 
validation  acquirements. 

ns-3 

The  ns-3  simulator  was  selected  because  it  is  actively 
developed  on  multiple  fronts,  written  entirely  in  C++, 
and  has  a  rich  set  of  WiMAX  modules.  Initial  funding  to 
develop  ns-3  was  provided,  in  part,  by  the  National 
Science  Foundation  in  2006.  Since  then,  the  ns-3 
development  effort  has  attracted  the  support  and  interest 
of  several  major  academic  departments  and  organizations 
including  the  Electrical  and  Computer  Engineering 
Department  at  the  Georgia  Institute  of  Technology,  the 
Electric  Engineering  Department  at  the  University  of 
Washington,  and  the  Google  Summer  of  Code. 

Ns-3  consists  of  a  simulation  core  engine,  a  set  of 
models,  example  programs,  and  tests.  The  ns-3  testing 
environment  provides  model  validation  and  testing  tools 
and  encourages  the  publication  of  validation  results. 
Characteristics  of  the  ns-3  development  effort  include 

•  strict  implementation  of  IEEE  specifications; 

•  broad  international  use  and  contribution; 

•  continuous  academic,  corporate,  and  public 
scrutiny  of  the  source  code; 

•  academic  “validation”  through  published  articles 
and  conference  presentations;  and 

•  extensive  testing. 
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A  search  on  the  Association  of  Computing  Ma¬ 
chinery  (ACM)  portal  for  articles  involving  the 
keyword,  “ns-3”,  identifies  nearly  500  articles.  Search¬ 
ing  with  the  additional  keyword,  “WiMAX”  yields  20 
articles  spanning  respected  conferences  such  as  SI- 
MUTools,  WICON,  SIGOPS,  SIGCOMM,  AAA- 
IDEA  Interperf,  and  ValueTools. 

It  is  difficult  to  guarantee  the  correctness  of  large- 
scale  software  simulators,  including  commercial  simu¬ 
lators.  Rather,  qualities  such  as  those  listed  above  build 
confidence  in  the  simulator’s  correctness,  including 
validation  and  verification. 

In  the  language  of  the  ns-3  documentation  (ns- 
developers@isi.edu  2010),  “ns-3  must  be  correct, 
robust,  performant  and  maintainable.”  In  summary, 
each  of  these  test  criteria  is  addressed  as  follows  (with 
excerpts  from  the  ns-3  testing  documentation): 

•  Correct:  “The  ns-3  testing  environment  provides 
tools  to  allow  for  both  model  validation  and 
testing,  and  encourages  the  publication  of 
validation  results.” 

•  Robust:  “The  ns-3  testing  environment  provides 
tools  to  allow  for  setting  up  and  running  test 
environments  over  multiple  systems  (buildbot)  and 
provides  classes  to  encourage  clean  tests  to  verify 
the  operation  of  the  system  over  the  expected 
“domain  of  applicability”  and  “range  of  accuracy.” 

•  Performant:  This  is  a  concise  neologism  that  is 
used  to  describe  the  design  goal  that  ns-3  must  be 
“powerful  and  fast  enough  to  get  the  job  done.  In 
the  ns-3  test  framework,  we  provide  support  for 
timing  various  kinds  of  tests.” 

•  Maintainable:  “The  ns-3  testing  framework 
provides  tools  for  automating  the  process  used 
to  validate  and  verify  the  code  in  nightly  test 
suites  to  help  quickly  identify  possible  regres¬ 
sions.”  These  regressions  include  local  regres¬ 
sions,  remote  regressions,  unmasked  regressions, 
and  performance  regressions. 

Radio  Mobile 

As  mentioned,  ns-3  lacks  a  GUI.  This  condition  not 
only  diminishes  usability  but  also  denies  the  user  the 
ability  to  graphically  specify  network  topologies.  A 
freeware  tool  called  Radio  Mobile  (RM)  (Coude  n.d.) 
exists  that  predicts  the  performance  of  a  radio  system 
by  using  digital  terrain  elevation  data.  RM  additionally 
provides  a  GUI  for  the  layout  of  wireless  network 
devices  on  top  of  a  rendered  topography.  The  output  of 
RM  can  be  used  to  configure  ns-3  for  more  realistic 
scenarios,  which  include  topology,  distance,  and  signal 
properties. 


RM  is  based  on  the  U.S.  Department  of  Commerce 
National  Telecommunications  and  Information  Ad¬ 
ministration  Institute  for  Telecommunication  Sciences 
(NTI A/ITS)  Irregular  Terrain  Model  (ITM)  (Long- 
ley-Rice).  This  software  has  been  in  use  since  1988. 
RM’s  publication  history  is  less  extensive  than  ns-3, 
with  only  a  few  articles  appearing  in  Radcom  (Brown 
2006)  and  AntenneX  (Brown  2009).  Comprehensive 
T&E  of  RM  is  not  present  in  the  literature.  However, 
our  use  of  RM  is  intended  to  more  accurately 
characterize  the  environment  in  which  our  TE  scenario 
will  be  deployed.  We  are  interested  in  the  following 
output  from  RM: 

•  topology  of  the  TE  scenario, 

•  accurate  distance  measures  between  BS  and  SS, 
and 

•  approximate  radio  propagation  qualities. 

We  have  validated  the  output  of  the  first  two  items 
by  inspection.  Furthermore,  RM  provides  a  three- 
dimensional  view  of  the  BS  and  SS  stations  as  well  as 
vectors  illustrating,  with  color,  the  signal  loss  as  a  result 
of  the  environment.  This  interface  provides  a  quick 
“sanity  check”  of  the  outputted  numerical  results.  Since 
the  built-in  ns-3  propagation  model  is  the  only  other 
environment  model  we  have  available,  the  reasonable 
approximation  of  radio  propagation  produced  by  RM 
is  a  significant  improvement. 

Gnuplot 

Gnuplot  is  an  open-source  cross-platform  com¬ 
mand-line-driven  graphing  utility.  It  was  created  for 
visualization  of  mathematical  functions  and  data 
interactively  and  has  been  under  active  development 
since  1986.  Gnuplot  is  used  extensively  in  the  scientific 
community.  A  search  on  the  ACM  portal  for  articles 
involving  the  keyword,  “gnuplot”,  identifies  nearly  300 
articles.  Our  use  of  gnuplot  is  for  the  two-dimensional 
visualization  of  performance  metrics  such  as  through¬ 
put. 

Net-Measure 

Net-Measure  (code.google  n.d.)  is  a  C++  class  that 
“wraps”  the  performance  monitoring  capability  of  ns-3 
(FlowMonitor)  into  an  easy-to-use  interface.  In 
addition,  Net-Measure  provides  interval-timed  cap¬ 
tures  of  network  metrics  so  that  performance  plots  can 
be  visualized  (with  gnuplot)  over  time.  The  imple¬ 
mentation  of  the  class  is  one  C++  file. 

As  Net-Measure  relies  on  the  correctness  of  the  ns-3 
FlowMonitor,  a  personal  review  of  the  single  source 
file  implementation  was  sufficient  to  verify  the 
correctness  of  the  implementation.  We  validated,  to 
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Figure  3.  Worldwide  Interoperability  for  Microwave  Access  (WiMAX)  platoon  configuration. 


our  satisfaction,  the  results  of  Net-Measure  on  three 
“built-in”  WiMAX  examples  within  ns-3. 

Python  script 

A  Python  script  is  utilized  to  translate  the  output 
results  of  RM  into  a  more  succinct  text  form  that  will 
be  imported  into  our  ns-3  simulation.  This  script  is 
short,  and  its  correctness  is  easily  verified  by  observing 
the  source  code.  We  validated  the  correctness  of  the 
results  by  comparing  the  translated  output  with  the 
input  from  RM. 

Integrated  solution  details 

Figure  2  illustrates  the  integration  and  coordination 
of  the  various  tools  described  in  the  previous  section. 
We  now  describe  the  flow  through  this  solution 
architecture  with  a  specific  TE  use  case. 

A  likely  deployed  isolated  squad  WiMAX  cell 
scenario  was  obtained  from  the  Communications- 
Electronics  Research,  Development,  and  Engineering 
Center  (CERDEC)  (see  Figure  3).  The  topology  of 
this  deployment  is  not  atypical  of  general  WiMAX 
topologies,  including  commercial  layouts.  CERDEC 
also  supplied  simulated  traffic;  characterizing  traffic 
types,  rates,  QoS  priorities,  and  timings  for  a  squad 
deployment. 

The  traffic  data  from  CERDEC  is  not  real  data,  but 
rather  representative  data  obtained  from  the  traffic 
generation  tool,  TGEN.  The  data  has  been  “sanitized” 
for  security  reasons  but  still  represents  a  realistic  TE 
traffic  flow  pattern  with  multiple  QoS  constraints, 


including  situational  awareness,  command  and  control, 
and  voice. 

Our  integrated  solution  commences  with  the  layout 
of  a  squad  topology  in  a  selected  region  using  RM  [1] 
as  shown  in  Figure  2.  The  output  of  RM  is  then 
converted  with  a  python  script  into  a  form  amenable  to 
ns-3  [2].  A  simulation  run  consisting  of  the  custom  use 
case  [8],  Net-Measure  module  [4],  and  a  command- 
line-specified  uplink  scheduler  [5]  is  initiated.  The 
plot  output  results  of  the  ns-3  simulation  are  then 
viewed  using  the  Gnuplot  utility  [7]. 

This  integrated  solution  is  not  unwieldy.  Since  we 
are  concerned  with  the  impact  of  the  uplink  scheduling 
algorithm  on  throughput  (with  QoS  constraints),  steps 
[1]  and  [2]  need  only  be  performed  once.  Components 
[3,  4,  5,  6,  and  8]  are  either  “hardwired”  into  the  code, 
or  automatically  configured/selected  as  a  result  of  the 
input.  Only  step  [7]  needs  to  be  performed  separately 
after  each  simulation  run. 

Results 

Figure  4  compares  the  performance  of  three  differ¬ 
ent  scheduling  algorithms.  The  graph  is  a  preliminary 
comparison  graph  we  constructed  utilizing  a  simplified 
simulation.  We  initially  coded  this  simple  simulator  to 
verify  the  results  of  one  particular  published  article 
(Mohammadi,  Akl,  and  Behnamfar  2008b),  which 
demonstrated  an  “optimal”  uplink  scheduling  algo¬ 
rithm  in  the  form  of  a  modified  0/1  Knapsack  problem. 

The  simple  simulation  compares  the  different 
scheduling  algorithms  in  a  theoretical  manner  in  that 
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Figure  4.  Simple  simulation  results. 

the  vast  majority  of  the  WiMAX  infrastructure  and 
protocols  are  not  modeled.  For  instance,  protocol 
stacks,  specific  application  data,  and  propagation 
characteristics  are  not  modeled.  The  simple  simulator 
consists  of  less  than  1,000  lines  of  code  and  is  typical  of 
many  simulations  that  appear  in  journal  articles 
comparing  WiMAX  scheduling  algorithms  (Moham- 
madi,  Akl,  and  Behnamfar  2008a,  2008b;  Wong- 
thavarawat  and  Ganz  2003). 

From  Figure  4,  we  see  that  as  the  number  of  packets 
waiting  to  be  sent  (queued)  at  the  SS  grows,  then  the 
different  algorithms  distinguish  themselves  by  per¬ 
forming  better  or  worse  in  terms  of  the  number  of 
packets  actually  sent.  Theoretically  speaking,  it  appears 
that  Algorithm  3  is  superior. 

The  TE-use  case  consists  of  a  set  of  nine  squad 
member  SS  and  one  control  center  BS  (see  Figure  3). 
In  this  figure,  the  BS  is  depicted  as  a  parabolic  antenna 
with  the  label  SQD_BS.  All  other  icons  on  the  map 
are  squad  members.  A  green  connecting  line  between 
the  BS  and  an  SS  indicates  a  high-quality  signal 
strength  radio  link.  The  distance  between  the  BS  and 
each  SS  ranges  from  14  km  to  32  km.  Each  SS  consists 
of  a  2-meter  high  omnidirectional  antenna  and  uses 
Binary  Phase  Shift  Keying  (BPSK)  1/2  modulation. 
Three  different  uplink  scheduling  algorithms  were 
evaluated. 

Simulation  results  using  the  TGEN  squad  traffic 
with  multiple  scheduling  algorithms  indicated  no 
difference  in  performance.  Figure  5  summarizes  the 
results  of  the  comparison  by  measuring  the  throughput 
over  time  for  traffic  between  two  arbitrary  nodes.  All 
three  uplink  algorithms  yielded  similar  throughput. 
Performance  between  other  pairs  of  nodes  behaved 
similarly. 

For  this  TE  scenario,  the  results  of  our  simulation 
indicate  that  any  uplink  scheduling  algorithm  can  be 
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Figure  5.  Net-measure  results. 

used  with  no  significant  change  in  performance.  These 
results  don’t  contradict  the  simple  simulation  results. 
Rather,  they  simply  indicate  that,  for  this  scenario,  the 
choice  of  uplink  scheduler  is  not  a  factor  on 
performance. 

We  note  that  the  nature  of  the  TGEN  squad  traffic 
is  fundamentally  multicast.  Though  ns-3  supports 
multicast  within  many  models,  the  documentation  is 
unclear  about  how  to  establish  multicast  groups 
attached  to  a  BS.  The  results  presented  here  substitute 
unicast  traffic  in  place  of  multicast  traffic.  Neverthe¬ 
less,  we  expect  the  results  to  be  the  same,  as  the  traffic 
volume  in  the  uplink  direction  would  not  increase 
significantly. 

Conclusion  and  future  work 

We  have  demonstrated  integrated  solution  architec¬ 
ture  for  T&E  of  WiMAX  performance  on  QoS- 
constrained  TE  traffic  using  open-source  modeling 
and  simulation  tools.  The  solution  is  general  in  that  it 
can  be  applied  to  other  WiMAX  performance- 
affecting  variables  such  as  modulation  type,  radio 
frequency,  terrain  profiles,  and  traffic  type. 

For  the  specific  TE  squad  use-case  we  investigated, 
the  choice  of  uplink  scheduling  algorithm  resulted  in  no 
detectable  degradation  in  performance.  Our  belief  is  that 
the  limited  volume  of  the  TGEN  squad  traffic  does  not 
saturate  the  available  bandwidth  in  the  uplink  direction. 
Thus,  algorithms  that  seek  to  optimize  bandwidth 
allocation  perform  no  better  than  simple  algorithms. 
With  larger-scale  scenarios  involving  more  loquacious 
network  applications,  we  anticipate  differences. 

We  identify  five  promising  avenues  for  continued 
work  using  this  architecture.  First,  the  characteristic 
multicast  nature  of  TE  communications  should  be 
modeled.  The  multicast  facility  is  present  in  ns-3  but 
requires  further  research  to  determine  how  to  use  it  at  a 
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WiMAX  BS.  Second,  larger-scale  simulations,  perhaps 
at  the  platoon  or  even  battalion  level,  can  be  pursued. 
Third,  WiMAX  TE  scenarios  involving  a  broader 
range  of  applications  exhibiting  greater  traffic  volume 
and  more  varied  QoS  constraints  can  considered.  Such 
a  set  of  applications  might  include  tactically  important 
text  chat,  white  board,  streaming  video,  and  applica¬ 
tion  sharing.  Fourth,  a  level  of  validation  can  be 
obtained  by  implementing  the  same  scenario  in 
another  simulator  (e.g.,  OPNET).  Fifth,  the  develop¬ 
ment  of  a  GUI  to  “seamlessly”  integrate  the  various 
open-source  and  freeware  tools  would  be  a  useful 
endeavor.1  □ 
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